Method for processing image data using a non-linear scaling model and a medical visual aid system

ABSTRACT

A method for obtaining and processing image data by using a medical visual aid system comprising an endoscope and a monitor is presented. The endoscope is configured to be inserted into a body cavity and comprises an image capturing device and a light emitting device. The method comprises illuminating a field of view of the image capturing device with the light emitting device, capturing the image data using the image capturing device, providing a non-linear scaling model adapted to the body cavity, adjusting the image data by applying the non-linear scaling model such that adjusted image data is formed, and presenting the adjusted image data on the monitor.

TECHNICAL FIELD

The present invention relates to endoscopes. More particularly, it isrelated to a method for obtaining and processing image data, a medicalvisual aid system, an endoscope forming part of the medical visual aidsystem and a monitor forming part of the visual aid system.

BACKGROUND

Endoscopes are well known devices for visually inspecting inaccessibleplaces such as human body cavities. Typically, the endoscope comprisesan elongated insertion tube with a handle at the proximal end as seenfrom the operator and visual inspections means, such as a built incamera with an image sensor and a light source at the distal end of theelongated insertion tube. The endoscopes are typically connected tomonitors in order to display images captured by the camera whileinserted into an object to be observed. Electrical wiring for the cameraand the light source such as a LED run along the inside of the elongatedinsertion tube from the handle to the tip at the distal end. Instead ofa LED endoscopes may also be fibre-optic, in which case the opticalfibres run along the inside of the elongated insertion tube.

In order to be able to manoeuvre the endoscope inside the body cavity,the distal end of the endoscope may comprise a section with increasedflexibility, e.g. an articulated tip part allowing the operator to bendthis section. Typically this is done by tensioning or slacking pullwires also running along the inside of the elongated insertion tube fromthe articulated tip part to a control mechanism of the handle.Furthermore, a working channel may run along the inside of the insertiontube from the handle to the tip, e.g. allowing liquid to be removed fromthe body cavity or allowing the insertion of surgical instruments or thelike into the body cavity.

In order to reduce the risk of cross-contamination and avoid thecumbersome procedure of cleaning endoscopes after use, it is desirableto provide endoscopes that are designed for single-use. In order to keepcosts at a low level single-use endoscopes are often designed with asfew as possible components. However, it is still desirable to obtain thebest possible image to be displayed on the screen. In complex reusableendoscopes one way of ensuring the image quality is achieved byproviding a light source that adequately illuminates the object to beobserved and the light intensity from the light source may even beautomatically adjusted by analysing images captured by the camera. Insingle-use endoscopes it is desirable with a simple light source such asan LED, which may be disposed at the distal tip without any opticalcomponents such as lenses, light guides or light reflecting elements tofocus, shape or distribute the light emitted from the LED. An example ofsuch configuration is known from WO14106511.

While this configuration is desirable due to the simple design, thelight source may course parts of the object to be observed to beover-exposured causing pixels of the image sensor to be saturated sothat information about the observed object is lost. As a result imagesdisplayed on the monitor will in some areas appear too bright and otherareas will appear too dark in order to derive the desired informationabout the object to be observed. This is especially the case when theendoscope is inserted into a tubular structure such as lungs of humanbeings.

With a view to this, the objective is to provide a method and anendoscope system that in a simple and cost efficient way improves theimage quality.

SUMMARY

Accordingly, the present invention preferably seeks to mitigate,alleviate or eliminate one or more of the above-identified deficienciesin the art and disadvantages singly or in any combination and solves atleast the above mentioned problems e.g. by providing according to afirst aspect a method for processing image data obtained using a medicalvisual aid system comprising an endoscope and a monitor, wherein theendoscope is configured to be inserted into a body cavity and comprisesan image capturing device for capturing image data and a light emittingdevice, said method comprising providing a non-linear scaling modeladapted to the body cavity, adjusting the image data by applying thenon-linear scaling model such that adjusted image data is formed,presenting the adjusted image data on the monitor

An advantage is that by using the non-linear scaling model and havingthis adapted to the body cavity, images, based on the adjusted imagedata, presented on the monitor can easily and quickly be analyzed by anoperator, which in turn implies improved health care.

The method may further comprise applying exposure settings of the imagecapturing device, such that no regions of the image covering more thantwo neighboring pixels, preferably more than a single pixel, aresaturated by light.

An advantage is that information will not be lost in overexposed areasof the image. The darkest areas of the image, may become even darker,but the information will still be in the pixels. Information in thedarkest area will be made visible to the user by application of thenon-linear scaling model.

The method may further comprise applying exposure settings emphasizingon a central part of the field of view of the image capturing device.

An advantage is, in case of examining a tube-formed cavity, that byemphasizing on the central part the image data reflecting this part willcomprise additional information, which in turn implies that a result ofa refinement of the image data, e.g. by using the non-linear scalingmodel, can be done at a later stage such that additional details aremade visible to the operator.

The non-linear scaling model may also be adapted to the monitor.

Different monitors may handle image data differently, and thereby byknowing which kind of monitor that is being used the non-linear scalingmodel can be adapted accordingly, which in turn results in that imagespresented on the monitor can easily be analyzed by the operator.

The non-linear scaling model may also be adapted to the light emittingdevice.

Different light emitting devices may enlighten the body cavitydifferently. Thus, by knowing which type of light emitting devices thatare being used by the endoscope the non-linear scaling model can beadapted, which in turn results in that images presented on the monitorcan easily be analyzed by the operator.

The non-linear scaling model may be a non-linear intensity scalingmodel, such as a non-linear gamma correction model.

The non-linear intensity scaling model may be configured to, in theimage displayed on the monitor, increase the contrast in the dark partsof the image and reduce the contrast in the parts of the image having anintermediate light intensity in a manner whereby pixels having low pixelintensity values are scaled up significantly and pixels having mid-rangepixel intensity values are only slightly adjusted or not adjusted atall.

Thus, the non-linear scaling model may provide a lower average gain tothe pixels having mid-range pixel intensity values than the average gainprovided by a standard gamma function, providing the same average gainto the pixels having low pixel intensity values as the non-linearscaling mode. The standard gamma function is defined as:

V _(out) =V _(in) ^(γ)

An advantage of this is that it may look like as if a light sourceilluminates both the regions close to the endoscope tip and the regionsfurther away from the endoscope tip with the same light intensity.

The dark part of the image may be defined as the parts of the imagehaving intensities between 0% and 7% of the maximum intensity. The partsof the image having an intermediate light intensity may be defined asthe parts of the image having intensities between 8% and 30% of themaximum intensity.

The non-linear intensity scaling model may be a scaling function mappinginput intensities to output intensities.

The scaling function may be provided with a bend, i.e. the slope of thescaling function may be neither continuously increasing or continuouslydecreasing. In some embodiments the scaling function has a first part,the first part being followed by a second part, the second part beingfollowed by a third part, and wherein the average slope of the secondpart is lower than the average slope of the first part and the averageslope of the third part.

This allows a high gain to be provided to the dark parts of the image, alower gain to be provided to the parts of the image having anintermediate light intensity, while at the same time utilizing the fulldynamic range of the monitor.

The step of adjusting the image data by applying the non-linearintensity scaling model such that adjusted image data is formed mayfurther comprise increasing intensity of a low intensity image datasub-set, wherein the low intensity image data sub-set comprises theimage data having intensity levels up to 25% of maximum intensity, anddecreasing intensity of a high intensity image data sub-set, wherein thehigh intensity image data sub-set comprises the image data havingintensity levels from 95% of the maximum intensity.

An advantage with this is that the adjusted image data that representsfaraway lying (in relation to the image capturing device) areas of thebody can easily be analyzed by the operator.

The non-linear scaling model may be set to increase intensity of the lowintensity image data sub-set by a first increase factor, wherein thefirst increase factor is greater than intensity factors used for othersub-sets of the image data.

An advantage of increasing the intensity of the low image intensityimage data sub-set to a higher extent than in the rest of the imagedata, is that faraway lying areas of the body cavity can easily beanalyzed by the operator.

The step of providing a non-linear scaling model adapted to the bodycavity may further comprise determining a body cavity type to which thebody cavity is related, and selecting the non-linear scaling model basedon the body cavity type.

An advantage of this is that differences in terms of shape and lightreflecting properties of different body cavities can be taken intoaccount, which in turn provides for that the non-linear scaling modelmay be customized for different body cavities, which in turn can make itpossible to provide images that can easily be analyzed via the monitorfor a wide range of different body cavities.

In some embodiments the non-linear scaling model used for adjusting theimage data is select from a set of non-linear scaling models comprisinga first non-linear scaling model and a second non-linear scaling model.In some embodiments the set comprises at least 3, at least 4 or at least5 non-linear scaling models.

In some embodiments the first non-linear scaling model and the secondnon-linear scaling model both are adapted to the same monitor.

In some embodiments the image data is obtained using a single useendoscope, and wherein the monitor adjusts the image data by applyingthe non-linear scaling model.

In some embodiments the first non-linear scaling model and the secondnon-linear scaling model are stored in the monitor.

According to a second aspect, a medical visual aid system comprising anendoscope and a monitor is provided, wherein the endoscope is configuredto be inserted into a body cavity and comprises an image capturingdevice and a light emitting device, and the monitor comprises an imagedata processing device for adjusting the image data by applying anon-linear scaling model adapted to the body cavity such that adjustedimage data is formed, and a display device for presenting the adjustedimage data.

An advantage is that by using the non-linear scaling model and havingthis adapted to the body cavity, images, based on the adjusted imagedata, presented on the monitor can easily and quickly be analyzed by anoperator, which in turn implies improved health care.

Further, exposure settings of the image data processing device may beconfigured to emphasize on a central part of the field of view of theimage capturing device.

An advantage is, in case of examining a tube-formed cavity, that byemphasizing on the central part the image data reflecting this part willcomprise additional information, which in turn implies that a result ofa refinement of the image data, e.g. by using the non-linear scalingmodel, can be done at a later stage such that additional details aremade visible to the operator.

The non-linear scaling model may also be adapted to the display deviceset as a recipient of the adjusted image data.

Different monitors may handle image data differently, and thereby byknowing which kind of monitor that is being used the non-scaling modelcan be adapted accordingly, which in turn results in that imagespresented on the monitor can easily be analyzed by the operator.

The non-linear scaling model may also be adapted to the light emittingdevice.

Different light emitting devices may enlighten the body cavitydifferently. Thus, by knowing which type of light emitting devices thatare being used by the endoscope the non-scaling model can be adapted,which in turn results in that images presented on the monitor can easilybe analyzed by the operator.

The non-linear scaling model may be a non-linear intensity scalingmodel, such as a non-linear gamma correction model.

The non-linear intensity scaling model may be configured to, in theimage displayed on the monitor, increase the contrast in the dark partsof the image and reduce the contrast in the parts of the image having anintermediate light intensity in a manner whereby pixels having low pixelintensity values are scaled up significantly and pixels having mid-rangepixel intensity values are only slightly adjusted or not adjusted atall.

Thus, the non-linear scaling model may provide a lower average gain tothe pixels having mid-range pixel intensity values than the average gainprovided by a standard gamma function, providing the same average gainto the pixels having low pixel intensity values as the non-linearscaling mode. The standard gamma function is defined as:

V _(out) =V _(in) ^(γ)

An advantage of this is that it may look like as if a light sourceilluminates both the regions close to the endoscope tip and the regionsfurther away from the endoscope tip with the same light intensity.

The dark part of the image may be defined as the parts of the imagehaving intensities between 0% and 7% of the maximum intensity. The partsof the image having an intermediate light intensity may be defined asthe parts of the image having intensities between 8% and 30% of themaximum intensity.

The non-linear intensity scaling model may be a scaling function mappinginput intensities to output intensities.

The scaling function may be provided with a bend, i.e. the slope of thescaling function may be neither continuously increasing or continuouslydecreasing. In some embodiments the scaling function has a first part,the first part being followed by a second part, the second part beingfollowed by a third part, and wherein the average slope of the secondpart is lower than the average slope of the first part and the averageslope of the third part. This allows a high gain to be provided to thedark parts of the image, a lower gain to be provided to the parts of theimage having an intermediate light intensity, while at the same timeutilizing the full dynamic range of the monitor.

The image data processing device for adjusting the image data byapplying a non-linear scaling model may be configured to increaseintensity of a low intensity image data sub-set, wherein the lowintensity image data sub-set comprises intensity levels up to 25% ofmaximum intensity, and decrease intensity of a high intensity image datasub-set, wherein the high intensity image data sub-set comprisesintensity levels from 95% of the maximum intensity.

An advantage with this is that the adjusted image data that representsfaraway lying (in relation to the image capturing device) areas of thebody can easily be analyzed by the operator.

The non-linear scaling model may be set to increase intensity of the lowintensity image data sub-set by a first increase factor, wherein thefirst increase factor is greater than intensity factors used for othersub-sets of the image data.

An advantage of increasing the intensity of the low image intensityimage data sub-set to a higher extent than in the rest of the image datais that faraway lying areas of the body cavity can easily be analyzed bythe operator.

The image data processing device may further be configured to determinea body cavity type to which the body cavity is related, and select thenon-linear scaling model based on the body cavity type.

An advantage of this is that differences in terms of shape and lightreflecting properties of different body cavities can be taken intoaccount, which in turn provides for that the non-linear scaling modelmay be customized for different body cavities, which in turn can make itpossible to provide images that can easily be analyzed via the monitorfor a wide range of different body cavities.

In some embodiments the non-linear scaling model used for adjusting theimage data is select from a set of non-linear scaling models comprisinga first non-linear scaling model and a second non-linear scaling model.

In some embodiments the first non-linear scaling model and the secondnon-linear scaling model both are adapted to the display device.

In some embodiments the image data is obtained using a single useendoscope.

In some embodiments the first non-linear scaling model and the secondnon-linear scaling model are stored in the monitor.

According to a third aspect, an endoscope configured to be inserted intoa body cavity is provided. This endoscope comprises an image capturingdevice and a light emitting device, and forms part of the medical visualaid system according to the second aspect.

According to a fourth aspect it is provided a monitor comprising animage data processing device for adjusting image data by applying anon-linear scaling model adapted to a body cavity such that adjustedimage data is formed, and a display device for presenting the adjustedimage data, and forming part of the visual aid system according to thesecond aspect.

According to a fifth aspect it is provided a computer program comprisingcomputer program code adapted to perform the method according to thefirst aspect when said computer program is run on a computer.

According to a sixth aspect is it provided a method for obtaining andpresenting image data by using a medical visual aid system comprising anendoscope and a monitor, wherein the endoscope is configured to beinserted into a body cavity and comprises an image capturing device anda light emitting device, said method comprising illuminating a field ofview of the image capturing device with the light emitting device,capturing the image data using the image capturing device, providing anon-linear scaling model adapted to the body cavity, adjusting the imagedata by applying the non-linear scaling model such that adjusted imagedata is formed, and presenting the adjusted image data on the monitor.

BRIEF DESCRIPTION OF DRAWINGS

The above, as well as additional objects, features and advantages of thepresent invention, will be better understood through the followingillustrative and non-limiting detailed description of preferredembodiments of the present invention, with reference to the appendeddrawings, wherein:

FIG. 1 illustrates an example of an endoscope.

FIG. 2 illustrates an example of a monitor that can be connected to theendoscope illustrated in FIG. 1.

FIG. 3 is a flow chart illustrating a number of steps for processingimage data before presenting this to an operator via e.g. the monitorillustrated in FIG. 2.

FIG. 4 generally illustrates a tip part of an endoscope placed inside abronchial tube.

FIG. 5 illustrates an example of an image depicting an inside of thebronchial tube captured by the endoscope.

FIG. 6 illustrates an example of a gamma correction model, which in turnis an example of a non-linear scaling model according to the invention.

FIG. 7a illustrates an example of image data before applying the gammacorrection model of FIG. 6.

FIG. 7b illustrates an example of image data after applying the gammacorrection model of FIG. 6.

FIG. 8 is a flowchart illustrating steps of a method for processingimage data before presenting on the monitor.

DETAILED DESCRIPTION

FIG. 1 illustrates an example of an endoscope 100. This endoscope may beadapted for single-use. The endoscope 100 is provided with a handle 102attached to an insertion tube 104 provided with a bending section 106.The insertion tube 104 as well as the bending section 106 may beprovided with one or several working channels such that instruments,such as a gripping device, may be inserted into a human body via theendoscope. One or several exit holes of the one or several channels maybe provided in a tip part 108 of the endoscope 100. In addition to theexit holes, a camera sensor, such as a CMOS sensor or any other imagecapturing device, as well as one or several light sources, such as lightemitting diodes (LEDs) or any other light emitting devices, may beplaced in the tip part 108. By having the camera sensor and the lightsources and a monitor 200, illustrated in FIG. 2, configured to displayimages based on image data captured by the camera sensor, an operator isable to see and analyze an inside of the human body in order to forinstance localize a position for taking a sample. In addition, theoperator will be able to control the instrument in a precise manner dueto the visual feedback made available by the camera sensor and themonitor. Further, since some diseases or health issues may result in ashift in natural colors or other visual symptoms, the operator isprovided with valuable input for making a diagnosis based on the imagedata provided via the camera sensor and the monitor.

In order to make it possible for the operator to direct the camerasensor such that different field of views can be achieved, the endoscopeis comprising a bending section 106 that can be bent in differentdirections with respect to the insertion tube 104. The bending section106 may be controlled by the operator by using a knob 110 placed on thehandle 102. The handle 102 illustrated in FIG. 1 is designed such thatthe knob 106 is controlled by a thumb of the operator, but other designsare also possible. In order to control a gripping device or other deviceprovided via a working channel a push button 112 may be used. The handle102 illustrated in FIG. 1 is designed such that a pointer finger of theoperator is used for controlling the gripping device, but other designsare also possible.

The image data captured by the camera sensor and optionally also otherdata captured by other sensors placed in the tip part can be transferredvia a connection cable 114 and a connector 116 to a monitor 200illustrated in FIG. 2. Even though wire-based data transmission isillustrated, it is equally possible to transfer image data by usingwireless data transmission.

The monitor device 200 is preferably a re-usable piece of equipment. Byhaving one single-use piece of equipment and another re-usable piece ofequipment, most of the data processing capability may be placed in there-usable piece of equipment in order to reach a cost efficient level atthe same time as being safe to use from a health perspective.

Data processing operations closely related to e.g. operation of thecamera sensor, such as reading out image data, may be performed in theendoscope itself, while more complex data processing operations,requiring more computational power, may be made in the monitor 200.Since most of the more complex data processing operations are related toimage data processing an Image Signal Processor (ISP) may be provided inthe monitor and used for image data processing operations.

In order to be able to display an image on the monitor, e.g. an imagethat is depicting the inside of the body being examined, such that theoperator easily can interpret this and make conclusions from it, theimage data captured by the image sensor can be processed in a number ofsteps. Below is given a sequence of steps 300, illustrated in FIG. 3, byway of example that can be made in order to be able to present the imagein a way such the operator easily can make use of this.

In a first step 302, the image data comprising pixels, each providedwith at least one pixel intensity value, is captured using an endoscope,such as the endoscope illustrated in FIG. 1.

In a second step 304, by using the ISP settings related to for instanceexposure may be adjusted in order to provide for that the image data iscaptured that at a later stage can be refined. If detected that theexposure settings are not adjusted correctly, for instance in that alarge portion of the image data is overexposed and/or underexposed, theexposure settings may be changed. This step may be performed in theendoscope.

In a third step 306, the image data can be demosaiced, e.g. conversionfrom Bayer level color space to RGB color space. For instance, in thisstep so-called Freeman interpolation (described in U.S. Pat. No.4,774,565) may be made in order to remove color artifacts in the imagedata. However, also other algorithms for demosaicing may be applied.

In a fourth step 308, a color temperature adjustment can be made. Thisadjustment may be based on user input. For instance, the operator mayhave chosen a color temperature scheme called “warm” comprising warmyellowish colors, and in this step the image data is adapted to complywith this color temperature scheme.

In a fifth step 310, as a complement to or instead of the colortemperature adjustment, independent gains may be set for the differentcolor channels, which in most cases are red, green and blue (RGB). Thissetting of gain may be based on user inputs.

In a sixth step 312, a gamma correction can be made. Gamma correction isin this context to be understood to be how much to increase or decreasepixel intensity values in different spans in order to provide for thatdetails in dark areas, i.e. low pixel intensity values, and light areas,i.e. high pixel intensity values, are made visible to the operator. Aswill be described in further detail below and further illustrated inFIG. 6, this step may be made by using a non-linear scaling model, e.g.a non-linear gamma correction model. Using the non-linear gammacorrection model has the effect that for instance a first pixel having alow pixel intensity value, though above zero, is significantly scaledup, that is, the low pixel intensity value is significantly increased,while a second pixel having mid-range pixel intensity value is onlyslightly adjusted or not adjusted at all. By using a non-linear scalingmodel in this way details of underexposed areas may become more easy toanalyze for the operator analyzing the image provided via the monitor200.

The non-linear scaling model may preferably be applied in connectionwith a controlling of the exposure settings of the image capturingdevice, such that over exposure, and thereby saturation of pixels, isavoided. If there is an area where the majority of pixels are saturated,information on the actual structure in this area is lost and cannot beregained. Preferably, the exposure settings are selected such that noregions of the image covering more than two neighboring pixels,preferably more than a single pixel, are saturated by light.

Even though illustrated to be performed after the third step 306, thegamma correction may in an alternative embodiment be made before thethird step 306 related to demosaicing.

In a seventh step 314, a color enhancement can be made. A consequence ofthe gamma correction made in the sixth step 312 can be that the imagedata loses its color intensity. Therefore, by having the colorenhancement made after the gamma correction, the color intensity may beadjusted after the gamma correction and hence compensate for effectscaused by the gamma correction. The adjustment may be made based on thata saturation gain is made, and how much gain to choose in this step maybe based on a gamma level used in the previous step, i.e. whichnon-linear gamma correction model that was used in the previous step incase several alternatives are available. For instance, if the gammalevel used in the previous step provided only a slight deviation from alinear gamma correction model a need for color enhancement may be lesscompared to if the gamma level used in the previous step significantlydeviated from the linear gamma correction model.

In an eighth step 316, a de-noising/sharpening can be made. This stepmay include both identifying noise caused by temporal effects, spatialeffects and signal-level effects, e.g. fixed pattern noise, and thenremoving this noise.

In a ninth step 318, the image data can be size scaled in order to reacha pre-set size, if different from current size.

In a tenth step 320, the image data can be transformed from one form toanother, in this particular example from a 10 bits format to a 8 bitsformat, before the image data is saved in an eleventh step 322.

In a twelfth step 324, the image data can be transformed from one formto another, in this particular example from a 10 bits format to a 6 bitsformat, before the image data is displayed in a thirteenth step 326.

The third to tenth step 306-320 and the twelfth step 324 may be made ina so-called FPGA (Field Programmable Gate Array) device provided in themonitor 200.

Even though exemplified in a certain order, different orders are alsopossible. In addition, one or several steps may be omitted if deemed notnecessary for instance based on a quality of the image data and/orrequirements of the images displayed on the monitor.

As illustrated in FIG. 4 by way of example, when examining a bronchialtube 400 using an endoscope 402, similar to the endoscope 100illustrated in FIG. 1, light sources 404 a, 404 b placed next to acamera sensor 406 may result in that a close lying first area 408 of thebronchial tube is lightened up significantly by the light sources 404 a,404 b, while a faraway lying second area 410 due to the tube-shaped formof the bronchial tube may not be lightened up at all, or at least lessthan the first area, by the light sources 404 a, 404 b. Thus, due to thetube-shaped form of the bronchial tube the first area 408 is likely tobe overexposed while the second area 410 is likely to be underexposed.

FIG. 5 illustrates an example of an image captured from a humanbronchial tube that clearly illustrates the effects presented inrelation to FIG. 4. An outer area 508 of the image, corresponding to theclose lying area 408 illustrated in FIG. 4, is overexposed, inparticular an upper right corner, and a central area 510, correspondingto the faraway lying area 410 illustrated in FIG. 4, is underexposed. Aneffect of having the overexposed and underexposed areas is that theoperator will not be able to analyze parts of the bronchial tubecorresponding to these areas.

In order to compensate for a shape and light reflecting properties of abody cavity, such as a bronchial tube as illustrated by way of examplein FIGS. 4 and 5, a non-linear gamma correction model may be used, asillustrated by way of example in FIG. 6.

As illustrated in FIG. 4, saturation of pixels is typically an issue onthe image part showing a proximal wall in a channel (in which theendoscope is operated) being close to the tip part of the endoscope.Such regions tend to be over-exposed. Not being able to see darkerdetails is an issue when looking deeper into the distal end of thechannel, i.e. regions further away from the tip of the endoscope andthereby further away from the light source.

This problem may be solved by implementing the following method. Whengrabbing the image data, e.g. under conditions as illustrated in FIG. 4,the sensitivity of the pixels is adjusted to a level where saturation ofpixels is unlikely to happen, even in image parts with high lightintensity. This adjustment can be performed by setting exposure time andgain (sensitivity) of the pixels of the image data.

This will have the result that the pixels in parts of the image wherethe light intensity is low will be even darker, and details in theseparts will not be discernible by the user when looking at the image assuch. However, information on the details in these parts will still bepresent, and by making a non-linear scaling, such as the non-lineargamma correction shown in FIG. 6, it is possible to process the imagesuch that details in both light parts and in dark parts can bediscernible to the user.

The non-linear scaling can be described with reference to FIG. 6,showing how a scaling can be performed from an original (In) image to ascaled (Out) image.

In a simple example, the pixels could be scaled one by one i.e. a pixelhaving a value x in the In image will obtain the value y in the Outimage. As indicated towards the black end of the In axis in FIG. 6, asmall change in x will cause a considerable larger change in y due tothe steep slope of the curve in darker regions. This will both cause anincreased contrast in the darker regions, and it will lift the level ofthe dark regions, i.e. these regions will get a higher light intensity.

The straight dashed line 600 in FIG. 6 is a one to one scaling, whichwill not change the image at all. The curved line 602 will, asmentioned, increase the contrast in the dark parts of the imageconsiderably, and make information in the darkest regions visible to theuser. At the same time, the contrast in regions having an intermediatelight intensity is reduced because of the relatively flat slope of thecurve. This has the purpose of not making the regions already beingrelatively light, too light for the user to discern details in theimage.

The effect of this non-linear contrast scaling is that the output imagepreferably should look as if a light source illuminates both the regionsclose to the endoscope tip and the regions further away from theendoscope tip with the same light intensity.

The exact shape of the curve will decide how the image will be scaled.In practice, curves of slightly different shapes will be provided, andthe user will have the option to select between different options in theuser interface, resulting in the use of one of these curves.Alternatively, the selection of curve could be made automatically, e.g.based on initial analysis of images from the body cavity being examined.

FIGS. 7a and 7b illustrates by way of example images depicting abronchial tube before and after the gamma correction model illustratedin FIG. 6 has been applied. In FIG. 7a , illustrating before the gammacorrection model has been applied, central portions 710 a, 710 b of theimage, i.e. parts of the bronchial tube lying relatively far away fromthe camera sensor, which corresponds to the second area 410 in FIG. 4,are dark and hence difficult for the operator to analyze. In FIG. 7b ,after the gamma correction model has been applied, these centralportions 710 c, 710 d are less dark, which has the positive effect thatthese can easily be analyzed.

Different body cavities are shaped in different ways. Thus, by havingdifferent gamma correction models for different body cavities, the imagedata can easily be transformed such that the operator more easily cananalyze this and make relevant conclusions based upon it. For instance,the gamma correction model illustrated in FIG. 6 may be used for imagedata captured in human body bronchial tubes. Which body cavity that isbeing examined may be provided as user input via e.g. the monitor, butit may also be automatically detected by the endoscope and/or themonitor itself e.g. by analyzing image data obtained by the imagecapturing device.

Transforming the image data by applying the non-linear gamma correctionmodel requires data processing capability. Since the endoscope may be asingle-use piece of equipment in order to assure that the risk ofcross-contamination between patients is eliminated, image dataprocessing operations, such as applying the gamma correction model, mayadvantageously be performed in the monitor, which is a re-usable pieceof equipment, in order to keep a total cost of operation low.

FIG. 8 generally illustrates a flowchart 800 illustrating a method forprocessing image data by using a medical visual aid system comprising anendoscope and a monitor. The endoscope is, as the endoscope illustratedin FIGS. 1 and 4, configured to be inserted into a body cavity andcomprises an image capturing device and a light emitting device, and themonitor is configured for presenting image data, as the monitorillustrated in FIG. 2. The method comprises four main steps.

In a first step 802 a field of view of the image capturing device isilluminated with the light emitting device.

In a second step 804, image data is captured using the image capturingdevice.

In a third step 806, a non-linear scaling model adapted to the bodycavity is provided.

In a fourth step 808, the image data is adjusted by applying thenon-linear scaling model such that adjusted image data is formed.

Again referring to FIG. 1, in order to reduce an impact of noise in theimage data transmitted from the image sensor it is known to shield theinsertion tube cable placed inside the insertion tube 102 and theconnection cable 114, in use provided between the endoscope 100 and themonitor 200. By doing so, a risk of having the image data affected bysignals from else-where can be reduced, which result in that there isless noise in the image data received by the monitor 200.

A disadvantage with shielding the insertion tube cable and theconnection cable is however increased production cost, and also sinceadditional material is needed an increased environmental cost. This ismore relevant for single-use endoscopes than for other equipment made tobe used multiple times.

Another disadvantage is that by having the insertion tube cable shieldeda flexibility of the insertion tube 102 may be decreased, which may showin that additional energy is needed for operating the bending section106 and/or that a range of the bending section 106 is limited due to theshielding.

Since at least a part of the noise is periodic noise or, in other words,fixed pattern noise, it is instead of shielding the insertion tube cableand/or the connection cable 114 a possibility to have the insertion tubecable and/or the connection cable 114 unshielded, or provided with areduced shielding, and to de-noise the image data in the monitor 200 inorder to remove, or at least reduce, the noise in the image data.

If the image sensor is read out line by line at regular intervals, theperiodic noise, for instance caused by the clock frequency, will bepresent in the image data with a regularity, which may show in thatthere is more noise in a first direction than in a second direction. Forinstance, the image data, which may be represented as a matrix with rowsand columns, may have a stronger presence of noise in a horizontaldirection, i.e. column noise, than in a vertical direction, i.e. rownoise.

Thus, in order to provide a cost efficient single-use endoscope withimproved flexibility it is possible to make use of a method forde-noising the image data as presented below.

A method for de-noising image data comprising first directional noise ina first direction and second directional noise in a second direction,wherein the first directional noise is greater than the seconddirectional noise, said method comprising

receiving the image data, comprising first pixels in the first directionand second pixels in the second direction, from the image sensorprovided in the tip part 108 of the endoscope 100, and

performing a convolution between the image data and a direction biasedperiodic noise compensation kernel, wherein the direction biasedperiodic noise compensation kernel is a convolution between an averagingkernel and a sharpening kernel, wherein the direction biased periodicnoise compensation kernel is providing equal averaging of the image datain the first direction and the second direction, and providingsharpening in the second direction.

The direction biased periodic noise compensation kernel may providesharpening in the second direction solely.

The first direction and the second direction may be perpendicular.

The endoscope 100 may comprise an insertion tube cable connected to theimage sensor, wherein the insertion tube cable may be unshielded.

The endoscope 100 may comprise a connection cable 114 connected to theinsertion tube cable, wherein the connection cable may be unshielded.

The step of performing the convolution between the image data and thedirection biased periodic noise compensation kernel may be made in themonitor 200 connected to the endoscope 100.

The direction biased periodic noise compensation kernel may be:

−1/3 −1/3 −1/3 1 1 1 −1/3 −1/3 −1/3

The averaging kernel may be:

1/9 1/9 1/9 1/9 1/9 1/9 1/9 1/9 1/9

The sharpening kernel may be:

−1 −1 −1 −1 9 −1 −1 −1 −1

The invention has mainly been described above with reference to a fewembodiments. However, as is readily appreciated by a person skilled inthe art, other embodiments than the ones disclosed above are equallypossible within the scope of the invention, as defined by the appendedpatent claims.

1. A method for processing image data obtained using a medical visualaid system comprising an endoscope and a monitor, wherein the endoscopeis configured to be inserted into a body cavity and comprises an imagecapturing device for capturing image data and a light emitting device,said method comprising: by the monitor: setting exposure settings togenerate darkened image data from the image capturing device, thedarkened image data including darker pixel areas and lighter pixelareas, the darker pixel areas being darker than the lighter pixel areas;receiving the darkened image data from the endoscope; accessing anon-linear intensity scaling model configured to change intensities ofthe pixels in the darker pixel areas more than intensities of the pixelsin the lighter pixel areas; applying the non-linear intensity scalingmodel to the darkened image data to form adjusted image data; andpresenting the adjusted image data.
 2. The method according to claim 1,wherein the exposure settings are configured such that no more than asingle pixel is saturated by light.
 3. The method according to claim 1,wherein the exposure settings are configured to emphasize a central partof a field of view of the image capturing device.
 4. The methodaccording to claim 1, wherein the non-linear intensity scaling model isalso adapted to the monitor.
 5. The method according to claim 1, whereinthe non-linear intensity scaling model is also adapted to the lightemitting device.
 6. (canceled)
 7. The method according to claim 1,wherein the non-linear intensity scaling model is configured to increasethe contrast in the dark parts of the darkened image data and reduce thecontrast in the parts of the image data having an intermediate lightintensity in a manner whereby pixels having low pixel intensity valuesare scaled up significantly and pixels having mid-range pixel intensityvalues are only slightly adjusted or not adjusted at all.
 8. The methodaccording to claim 7, wherein the non-linear intensity scaling model isa scaling function mapping input intensities to output intensities. 9.The method according to claim 8, wherein the scaling function includes abend.
 10. The method according to claim 9, wherein the scaling functionhas a first part, the first part being followed by a second part, thesecond part being followed by a third part, and wherein the averageslope of the second part is lower than the average slope of the firstpart and the average slope of the third part.
 11. The method accordingto claim 1, wherein the darkened image data includes a low intensityimage data sub-set, and a high intensity image data sub-set, wherein thelow intensity image data sub-set comprises pixels having intensitylevels up to 25% of a maximum intensity, and wherein the high intensityimage data sub-set comprises pixels having intensity levels from 95% ofthe maximum intensity, wherein the non-linear intensity scaling model isconfigured to increase the intensities of the pixels forming the lowintensity image data sub-set and to decrease the intensities of thepixels forming the high intensity image data sub-set.
 12. The methodaccording to claim 11, wherein the darkened image data includes anintermediate intensity image data sub-set, wherein the non-linearintensity scaling model is set to increase intensity of the lowintensity image data sub-set by a first increase factor and to increaseintensity of the intermediate intensity image data sub-set by a secondincrease factor, wherein the first increase factor is greater than thesecond increase factor.
 13. The method according claim 1, whereinaccessing a non-linear intensity scaling model comprises: determining abody cavity type to which the body cavity is related, and selecting thenon-linear intensity scaling model based on the body cavity type. 14-17.(canceled)
 18. A medical visual aid system comprising a monitor adaptedto communicate with an endoscope, wherein the endoscope is configured tobe inserted into a body cavity and comprises an image capturing deviceand a light emitting device, and the monitor comprises an image dataprocessing device for adjusting the image data by applying a non-linearscaling model adapted to the body cavity such that adjusted image datais formed, and a display device for presenting the adjusted image data.19. The medical visual aid system according to claim 18, whereinexposure settings of the image data processing device are configured toemphasize on a central part of a field of view of the image capturingdevice.
 20. (canceled)
 21. The medical visual aid system according toclaim 18, wherein the non-linear scaling model is also adapted to thelight emitting device.
 22. The medical visual aid system according toclaim 18, wherein the non-linear scaling model is a non-linear intensityscaling model.
 23. The medical visual aid system according to claim 22,wherein the non-linear intensity scaling model is configured to, in theimage presented on the display device, increase the contrast in the darkparts of the image and reduce the contrast in the parts of the imagehaving an intermediate light intensity in a manner whereby pixels havinglow pixel intensity values are scaled up significantly and pixels havingmid-range pixel intensity values are only slightly adjusted or notadjusted at all.
 24. The medical visual aid system according to claim23, wherein the non-linear intensity scaling model is a scaling functionmapping input intensities to output intensities.
 25. The medical visualaid system according to claim 24, wherein the scaling function isprovided with a bend.
 26. The medical visual aid system according toclaim 25, wherein the scaling function has a first part, the first partbeing followed by a second part, the second part being followed by athird part, and wherein the average slope of the second part is lowerthan the average slope of the first part and the average slope of thethird part.
 27. The medical visual aid system according to claim 22,wherein the image data processing device for adjusting the image data byapplying a non-linear scaling model is configured to increase intensityof a low intensity image data sub-set, wherein the low intensity imagedata sub-set comprises intensity levels up to 25% of maximum intensity,and decrease intensity of a high intensity image data sub-set, whereinthe high intensity image data sub-set comprises intensity levels from95% of the maximum intensity.
 28. The medical visual aid systemaccording to claim 18, wherein the non-linear scaling model is set toincrease intensity of the low intensity image data sub-set by a firstincrease factor, wherein the first increase factor is greater than theincrease factors used for other sub-sets of the image data.
 29. Themedical visual aid system according to claim 18, wherein the image dataprocessing device is further configured to determine a body cavity typeto which the body cavity is related, and select the non-linear scalingmodel based on the body cavity type. 30-33. (canceled)
 34. The medicalvisual aid system according to claim 18, further comprising theendoscope.
 35. (canceled)
 36. A computer program comprising computerprogram code adapted to perform a method when said computer program isrun on a computer, said method comprising: receiving image datacorresponding to a body cavity from an endoscope, the image dataincluding saturated pixels and unsaturated pixels; setting exposuresettings to generate darkened image data from the image data, thedarkened image data including darker pixel areas and lighter pixelareas, the darker pixel areas being darker than the lighter pixel areas;accessing a non-linear intensity scaling model configured to changeintensities of the pixels in the darker pixel areas more thanintensities of the pixels in the lighter pixel areas; applying thenon-linear intensity scaling model to the darkened image data to formadjusted image data; and outputting the adjusted image data.
 37. Themethod of claim 1, wherein the monitor has stored therein a plurality ofnon-linear intensity scaling models, and accessing a non-linearintensity scaling model comprises receiving, with a user interface, aselection by a user of the non-linear intensity scaling model from theplurality of non-linear intensity scaling models.
 38. The method ofclaim 1, further comprising, by the endoscope: generating the darkenedimage data by reducing an exposure in the image capturing device basedon the exposure settings set by the monitor.
 39. The method of claim 38,wherein the exposure settings are based on an evaluation of precedingimage data.